在本文中，我们介绍了一种新的基于GNN的知识图形嵌入模型，命名为WGE，以捕获聚焦的图形结构和关联的图形结构。特别是，鉴于知识图形，WGE构建一个无向实体的聚焦图，该图形将实体视为节点。此外，WGE还从关联的约束构造另一个无向图形，将实体和关系视为节点。然后，WGE提出了一种新的架构，即直接在这两个单个图表上使用两个vanilla GNNS，以更好地更新实体和关系的矢量表示，然后是加权得分函数来返回三重分数。实验结果表明，WGE在三个新的和具有挑战性的基准数据集Codex上获得最先进的表演，用于知识图形完成。

自动编辑后（APE）是减少通过机器翻译（MT）系统或软件辅助翻译产生的原始翻译文本错误的重要补救措施。在本文中，我们提出了一种系统的方法来解决越南人的APE任务。具体来说，我们构建了5M越南翻译和纠正句对的第一个大规模数据集。然后，我们使用由构造的数据集应用强大的神经MT模型来处理APE任务。自动和人类评估的实验结果表明了神经MT模型在处理越南APE任务方面的有效性。

我们介绍了一个名为Nuge的新型嵌入式模型，旨在将实体和关系之间的共同发生整合到图形神经网络中，以改善知识图形完成（即，链接预测）。鉴于知识图形，Nuge将单个图形构建，考虑实体和关系作为单个节点。然后，Nuge基于实体和关系的共同发生来计算节点之间的边缘的权重。接下来，Nuge提出双季型图形神经网络（DualQGNN），并利用DualQGNN更新实体和关系节点的向量表示。然后采用分数函数来产生三重分数。综合实验结果表明，NOGE在三个新的和困难的基准数据集Codex上获得最先进的结果，用于知识图形完成。

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Graph neural networks (GNNs) have demonstrated excellent performance in a wide range of applications. However, the enormous size of large-scale graphs hinders their applications under real-time inference scenarios. Although existing scalable GNNs leverage linear propagation to preprocess the features and accelerate the training and inference procedure, these methods still suffer from scalability issues when making inferences on unseen nodes, as the feature preprocessing requires the graph is known and fixed. To speed up the inference in the inductive setting, we propose a novel adaptive propagation order approach that generates the personalized propagation order for each node based on its topological information. This could successfully avoid the redundant computation of feature propagation. Moreover, the trade-off between accuracy and inference latency can be flexibly controlled by simple hyper-parameters to match different latency constraints of application scenarios. To compensate for the potential inference accuracy loss, we further propose Inception Distillation to exploit the multi scale reception information and improve the inference performance. Extensive experiments are conducted on four public datasets with different scales and characteristics, and the experimental results show that our proposed inference acceleration framework outperforms the SOTA graph inference acceleration baselines in terms of both accuracy and efficiency. In particular, the advantage of our proposed method is more significant on larger-scale datasets, and our framework achieves $75\times$ inference speedup on the largest Ogbn-products dataset.

Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.

我们介绍了第一项经验研究，研究了突发性检测对意向检测和插槽填充的下游任务的影响。我们对越南人进行了这项研究，这是一种低资源语言，没有以前的研究，也没有公共数据集可用于探索。首先，我们通过手动添加上下文不满并注释它们来扩展流利的越南意图检测和插槽填充phoatis。然后，我们使用强基线进行实验进行实验，以基于预训练的语言模型，以检测和关节意图检测和插槽填充。我们发现：（i）爆发对下游意图检测和插槽填充任务的性能产生负面影响，并且（ii）在探索环境中，预先训练的多语言语言模型XLM-R有助于产生更好的意图检测和插槽比预先训练的单语言模型phobert填充表演，这与在流利性环境中通常发现的相反。

数十年来，计算机系统持有大量个人数据。一方面，这种数据丰度允许在人工智能（AI），尤其是机器学习（ML）模型中突破。另一方面，它可能威胁用户的隐私并削弱人类与人工智能之间的信任。最近的法规要求，可以从一般情况下从计算机系统中删除有关用户的私人信息，特别是根据要求从ML模型中删除（例如，“被遗忘的权利”）。虽然从后端数据库中删除数据应该很简单，但在AI上下文中，它不够，因为ML模型经常“记住”旧数据。现有的对抗攻击证明，我们可以从训练有素的模型中学习私人会员或培训数据的属性。这种现象要求采用新的范式，即机器学习，以使ML模型忘记了特定的数据。事实证明，由于缺乏共同的框架和资源，最近在机器上学习的工作无法完全解决问题。在本调查文件中，我们试图在其定义，场景，机制和应用中对机器进行彻底的研究。具体而言，作为最先进的研究的类别集合，我们希望为那些寻求机器未学习的入门及其各种表述，设计要求，删除请求，算法和用途的人提供广泛的参考。 ML申请。此外，我们希望概述范式中的关键发现和趋势，并突出显示尚未看到机器无法使用的新研究领域，但仍可以受益匪浅。我们希望这项调查为ML研究人员以及寻求创新隐私技术的研究人员提供宝贵的参考。我们的资源是在https://github.com/tamlhp/awesome-machine-unlearning上。

在本文中，我们介绍了一个高质量的大规模基准数据集，用于英语 - 越南语音翻译，其中有508音频小时，由331k的三胞胎组成（句子长度的音频，英语源笔录句，越南人目标subtitle句子）。我们还使用强基础进行了经验实验，发现传统的“级联”方法仍然优于现代“端到端”方法。据我们所知，这是第一个大规模的英语 - 越南语音翻译研究。我们希望我们的公开数据集和研究都可以作为未来研究和英语语音翻译应用的起点。我们的数据集可从https://github.com/vinairesearch/phost获得

我们考虑了最小化客观功能的优化问题，该问题允许变异形式，并根据\ textIt {约束域}上的概率分布定义，这对理论分析和算法设计构成了挑战。受镜下降算法的启发，我们提出了一种迭代和基于粒子的算法，称为镜像变异传输（\ textbf {mirriryvt}）。对于每次迭代，\ textbf {mirrirvt}将粒子映射到由镜像映射引起的无约束的双空间，然后大约在通过推动粒子来定义的分布的歧管上大致执行wasserstein梯度下降。在迭代结束时，将粒子映射回原始的约束空间。通过模拟实验，我们证明了\ textbf {mirrirvt}的有效性，可以最大程度地限制函数，而不是单纯形和欧几里得球受到的域上的概率分布。我们还分析了其理论特性，并将其融合到目标功能的全局最小值。